HSEARCH-4674 Use test containers #3796

marko-bekhta · 2023-10-19T16:45:46Z

https://hibernate.atlassian.net/browse/HSEARCH-4674

the testcontainer+dockerfile+dependabot combination seems to work 🙈 😃
Here're a few PRs opened for DB updates:
marko-bekhta#190
marko-bekhta#188

while I was testing it on my repo I've added:

    username: ${{secrets.MY_DOCKERHUB_USER}}
    password: ${{secrets.MY_DOCKERHUB_TOKEN}}

we probably would need to add some secrets (as I'm not sure it works without the registry)

There's still some cleanup needed, but wanted to see if this still would work on CI

marko-bekhta · 2023-10-23T14:51:52Z

Added a test containers BOM and a couple of comments. aaand converting it from the draft 😃

yrodiere

Thanks, here are a few comments.

On the static containers... it's not a hard no, I just feel uncomfortable with something that is so widely frowned upon. If this is standard practice when using test containers, then so be it. But I'd really have felt safer with a solution leveraging JUnit contexts where possible.

build/parents/build/pom.xml

.github/dependabot.yml

build/parents/integrationtest/pom.xml

...c/main/java/org/hibernate/search/util/impl/integrationtest/mapper/orm/DatabaseContainer.java

pom.xml

integrationtest/java/modules/orm-elasticsearch/src/main/resources/actual-hibernate.properties

...c/main/java/org/hibernate/search/util/impl/integrationtest/mapper/orm/DatabaseContainer.java

.../src/main/java/org/hibernate/search/util/impl/integrationtest/mapper/orm/OrmSetupHelper.java

yrodiere

Some notes about properties mentioned in CONTRIBUTING.md.

build/parents/integrationtest/pom.xml

...va/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/SearchContainer.java

.github/dependabot.yml

marko-bekhta · 2023-10-24T13:24:02Z

There're more changes to come ... I just wanted to test Ruyk on CI 🙈 😃.

...c/main/java/org/hibernate/search/util/impl/integrationtest/mapper/orm/DatabaseContainer.java

build/container/oracle.Dockerfile

...lper/orm/src/test/java/org/hibernate/search/test/batchindexing/DatabaseMultitenancyTest.java

yrodiere

LGTM overall, here are my latest comments...

Did you check how execution time compares to what we had previously with docker-maven-plugin?

build/parents/build/pom.xml

build/parents/integrationtest/pom.xml

.github/workflows/build.yml

CONTRIBUTING.md

build/parents/integrationtest/pom.xml

pom.xml

marko-bekhta · 2023-10-31T08:44:40Z

the execution time is more or less the same compared to the maven docker plugin:

Env	Test Containers	Maven Docker plugin
Opensearch	11m 38s	12m 42s
Oracle	15m 24s	12m 40s
Postgres	9m 53s	9m 25s
Default build ITs	25m 38s	25m 16s

I'll take a look at Oracle tests and why it took that much longer:
Oracle Testcontaienrs:
Hibernate Search ITs - ORM - Jakarta Batch ......... SUCCESS [04:26 min]
Oracle maven-docker-plugin:
Hibernate Search ITs - ORM - Jakarta Batch ......... SUCCESS [02:24 min]

But since previous builds show a time close to 3 min, maybe it's just the change from XE to FREE version of the Oracle container...

yrodiere

Oh my, I just tried from the IDE: I just change the version in the ~~Jenkinsfile~~ Dockerfile (obviously), run the test and BOOM: a container gets started automatically with the right version. It's awesome.

I added a few comments below, could you please process them?

I also pushed a commit to your branch. The changes are very minor, but please take care to keep it when you rebase :)

Then once you're done rebasing on main (there's a conflict), and maybe after you've squashed a few commits if it makes sense, feel free to merge :)

.github/dependabot.yml

.github/hibernate-github-bot.yml

CONTRIBUTING.md

build/container/elastic.Dockerfile

build/container/amazon-opensearch-serverless.Dockerfile

build/container/opensearch.Dockerfile

...hibernate/search/util/impl/integrationtest/backend/elasticsearch/SearchBackendContainer.java

…ners

- remove some unused/redundant config properties - make use of @DynamicPropertySource - make use of `Persistence.createEntityManagerFactory` with properties - make use of `SystemHelper.setSystemProperty`

yrodiere

LGTM, please go ahead and squash/fixup what's needed and merge!
Thanks for all this :)

.github/dependabot.yml

…itHub Actions workflows

sonarcloud · 2023-11-08T08:27:57Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

marko-bekhta mentioned this pull request Oct 19, 2023

HSEARCH-3654 HSEARCH-4674 Upgrade to JUnit 5 + TestContainers #3320

Closed

marko-bekhta force-pushed the test/HSEARCH-4674-use-test-containers branch 3 times, most recently from d212435 to 8f99544 Compare October 23, 2023 14:50

marko-bekhta marked this pull request as ready for review October 23, 2023 14:51

yrodiere reviewed Oct 23, 2023

View reviewed changes

yrodiere reviewed Oct 24, 2023

View reviewed changes

build/parents/integrationtest/pom.xml Show resolved Hide resolved

build/parents/integrationtest/pom.xml Outdated Show resolved Hide resolved

yrodiere reviewed Oct 24, 2023

View reviewed changes

...va/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/SearchContainer.java Outdated Show resolved Hide resolved

.github/dependabot.yml Outdated Show resolved Hide resolved

marko-bekhta force-pushed the test/HSEARCH-4674-use-test-containers branch from 8f99544 to 0fd2fe8 Compare October 24, 2023 13:23

marko-bekhta force-pushed the test/HSEARCH-4674-use-test-containers branch 12 times, most recently from c78d01c to 177dd6e Compare October 25, 2023 13:42

marko-bekhta commented Oct 26, 2023

View reviewed changes

...c/main/java/org/hibernate/search/util/impl/integrationtest/mapper/orm/DatabaseContainer.java Show resolved Hide resolved

marko-bekhta commented Oct 26, 2023

View reviewed changes

build/container/oracle.Dockerfile Outdated Show resolved Hide resolved

marko-bekhta commented Oct 26, 2023

View reviewed changes

...lper/orm/src/test/java/org/hibernate/search/test/batchindexing/DatabaseMultitenancyTest.java Show resolved Hide resolved

yrodiere reviewed Oct 30, 2023

View reviewed changes

marko-bekhta force-pushed the test/HSEARCH-4674-use-test-containers branch from ddbb3ad to 0da07c6 Compare October 30, 2023 16:26

marko-bekhta mentioned this pull request Oct 30, 2023

HSEARCH-4674 DO NOT MERGE #3810

Closed

yrodiere approved these changes Nov 7, 2023

View reviewed changes

yrodiere force-pushed the test/HSEARCH-4674-use-test-containers branch from 83ab4f6 to b43bcf0 Compare November 7, 2023 12:55

marko-bekhta added 14 commits November 7, 2023 17:09

HSEARCH-4674 Use test containers for Elasticsearch/OpenSearch

84da276

HSEARCH-4674 Use test containers for databases

51f0eb9

HSEARCH-4674 Remove maven docker plugin

455a68c

HSEARCH-4674 Update groovy script to list containers

3437731

HSEARCH-4674 Use image name/tag defined in pom files to start containers

a190982

HSEARCH-4674 Remove some redundant properties

eeef0e2

HSEARCH-4674 Change how spring receives properties for test execution

ad273a0

HSEARCH-4674 Make batch test that doesn't use helpers aware of contai…

36a3b09

…ners

HSEARCH-4674 Make showcase and v5 tests aware of testcontainers

12678bc

HSEARCH-4674 Make java modules tests aware of testcontainers

8a8ff28

HSEARCH-4674 Make documentation tests aware of testcontainers

bf018a3

HSEARCH-4674 Use Dockerfiles to create database test containers

4a603cd

HSEARCH-4674 Add dependabot config for dockerfiles

8468eb1

HSEARCH-4674 Simplify tests configuration

4422c26

- remove some unused/redundant config properties - make use of @DynamicPropertySource - make use of `Persistence.createEntityManagerFactory` with properties - make use of `SystemHelper.setSystemProperty`

marko-bekhta force-pushed the test/HSEARCH-4674-use-test-containers branch 2 times, most recently from 7d9faad to 777c47f Compare November 7, 2023 16:47

yrodiere approved these changes Nov 8, 2023

View reviewed changes

yrodiere reviewed Nov 8, 2023

View reviewed changes

.github/dependabot.yml Show resolved Hide resolved

marko-bekhta force-pushed the test/HSEARCH-4674-use-test-containers branch from 777c47f to 3717b39 Compare November 8, 2023 07:31

marko-bekhta and others added 6 commits November 8, 2023 08:51

HSEARCH-4674 Enable reusable testcontainers on CI

e264052

HSEARCH-4674 Rework Elasticsearch/OpenSearch container integration

a2ef989

HSEARCH-4674 Allow custom wait conditions for database containers

642cf6b

HSEARCH-4674 Switch to IBM registry for DB2

bf4343b

HSEARCH-4674 Update comments on TestRunner classes

8918af4

HSEARCH-4674 Use groups for Docker updates in Dependabot config and G…

9860481

…itHub Actions workflows

marko-bekhta force-pushed the test/HSEARCH-4674-use-test-containers branch from f12b629 to 9860481 Compare November 8, 2023 07:52

marko-bekhta merged commit 16a0d19 into hibernate:main Nov 8, 2023
12 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HSEARCH-4674 Use test containers #3796

HSEARCH-4674 Use test containers #3796

marko-bekhta commented Oct 19, 2023

marko-bekhta commented Oct 23, 2023

yrodiere left a comment

yrodiere left a comment

marko-bekhta commented Oct 24, 2023

yrodiere left a comment

marko-bekhta commented Oct 31, 2023

yrodiere left a comment •

edited

Loading

yrodiere left a comment

sonarcloud bot commented Nov 8, 2023

HSEARCH-4674 Use test containers #3796

HSEARCH-4674 Use test containers #3796

Conversation

marko-bekhta commented Oct 19, 2023

marko-bekhta commented Oct 23, 2023

yrodiere left a comment

Choose a reason for hiding this comment

yrodiere left a comment

Choose a reason for hiding this comment

marko-bekhta commented Oct 24, 2023

yrodiere left a comment

Choose a reason for hiding this comment

marko-bekhta commented Oct 31, 2023

yrodiere left a comment • edited Loading

Choose a reason for hiding this comment

yrodiere left a comment

Choose a reason for hiding this comment

sonarcloud bot commented Nov 8, 2023

yrodiere left a comment •

edited

Loading